Utility based inquiry selection in a streaming data pipeline

ABSTRACT

Systems and methods disclosed herein relate to utility based selection of inquiries in a streaming data pipeline. An indication of user identification data and a data source may be received by a server in communication with a distributed streaming data pipeline. This pipeline may be in communication with asynchronous event processing services. Those event processing services may determine an inquiry rule that includes an inquiry threshold. This rule may be based on the data source and retrieval of information related to that data source from memory in a database server. The rule may prompt a comparison of an inquiry count to an inquiry threshold. If the count is below the threshold, an inquiry may be sent to a user based in part upon expected utility that is itself based upon several indications of expected utility calculated by an event processing service.

TECHNICAL FIELD

This application relates to selections of inquiries based upon adetermination of utility, and more particularly to implementation ofsuch features using a streaming data pipeline in communication withasynchronous event processing services.

BACKGROUND

The digital marketing industry uses various types of data to determinethe best manner to provide advertising to and to gather information fromvarious users. This data may include a user's age, ethnicity, income,location and other easily observable or collectable data.

Some companies create focused websites to drive users into their sitesfor collection of data and sales of advertising without using theinventive features taught below.

The prior art data collection methods collect or generate generallyapplicable user data, but often lack the ability to collect or generatespecifically targeted data. Without such data, determining the utilitybehind providing a user with a particular opportunity or collectingfurther information from the user may provide a less accurate result.For example, if the prior methods are not able to determine whether auser has a particular disease or condition, then such methods cannotdetermine the utility associated with determining whether the user maywish to treat the disease/condition. And if the methods cannot determineeither the presence of the disease/condition or the desire to treat thedisease/condition, then the methods cannot determine the utilityassociated with determining whether a user prefers a daily pill orweekly injection as a treatment method. Hypothetically, a skin disordermay be present in some subset of the population and utility may exist indetermining whether the population having the disorder is interested intreating the skin disorder. Extending the hypothetical, if the skindisorder can be treated either by taking a pill daily or by givingoneself a weekly injection, utility may exist in determining whatportion of the population and/or which specific user having the disorderwould prefer the weekly injection as opposed to the daily pill.Competing with this, utility may be found in determining other features,issues, preferences, or desires with respect to the same user. Forexample, utility may be present in determining whether the user desiresto vacation in sunny locations. And the utility of obtaining suchinformation may change after determining whether the user suffers fromthe skin disorder. However the user may have a tolerance for onlyproviding a limited set of information. So after determining that theuser has the skin disorder, it may be necessary to determine whether agreater utility exists in gathering information regarding treatmentpreferences versus vacation preferences. The known prior art systems donot adequately handle this determination in a manner that increasesoverall utility.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B illustrate an example flow diagram for implementing andusing the inventive system and method;

FIG. 2 illustrates an example of data flow and data processing that maybe implemented in the inventive system and method;

FIG. 3 illustrates an example of data flow and data processing that maybe used for suppression and verification;

FIG. 4 illustrates an example of data flow and data processing that maybe used for account or campaign administration;

FIG. 5 illustrates an example of data flow and data processing that maybe implemented in the inventive system and method;

FIGS. 6A, 6B, and 6C illustrate an exemplary layout for capturinginformation related to the publishers which an advertiser wishes totarget;

FIGS. 7A and 7B illustrate an exemplary layout for capturing bids andproviding statistics related to bids;

FIGS. 8A and 8B illustrate an exemplary layout for capturing and editingpublisher settings;

FIGS. 9A and 9B illustrate an exemplary layout for capturing and editingcampaign settings;

FIG. 10 illustrates an example block diagram of a computer operable toexecute the disclosed architecture; and

FIG. 11 illustrates an example schematic block diagram for a computingenvironment in accordance with the subject specification.

DETAILED DESCRIPTION

The various embodiments are now described with reference to thedrawings, wherein like reference numerals are used to refer to likeelements throughout. In the following description, for purposes ofexplanation, numerous specific details are set forth in order to providea thorough understanding of the various embodiments. It may be evident,however, that the various embodiments can be practiced without thesespecific details. In other instances, well-known structures and devicesare shown in block diagram form to facilitate describing the variousembodiments.

The following description and the drawings set forth certainillustrative aspects of the specification. These aspects are indicative,however, of but a few of the various ways in which the principles of thespecification may be employed. Other advantages and novel features ofthe specification will become apparent from the following detaileddescription of the specification when considered in conjunction with thedrawings.

Rather than relating to standard data collection as indicated in thebackground section, the invention permits the collection and use of newclasses of data and inventive methods of collecting such data. Ratherthan being tied to standard demographic information, the inventivemethod relates to both the collection of data regarding historicallynon-observable information about a user that may relate to desires orinterests of the user that cannot necessarily be determined based upontraditional data collection methods. For example, the invention maypermit collection of data regarding a user's favorite color, preferencesfor pet ownership, interest in a particular type of automobile,motivation to change service providers for a service being consumed,interest in obtaining further education, etc.

The invention employs an intelligent decision engine includingappropriate algorithms and models in a technical environment thatincreases processing speed and decreases processing time to retain alarger percentage of users through the data collection process. Theinventive system and method may include a data discovery platform thatmay be Internet-based and that permits dynamic service of inquiries tousers, for example, website visitors and mobile application users, in anattempt to find data. Collected data may then be used to deliver a userexperience, product, or service. One such user experience may be anadditional inquiry. Another such user experience may be an advertisementor advertising campaign. Many types of user experience may be deliveredwithout departing from the scope of the inventive system.

In many embodiments of the inventive system, it is anticipated that fourprimary groups of actors are involved. These groups include publishers,advertisers, users, and an implementer of the inventive system. Whileapplicants do not wish to define these terms narrowly, it is anticipatedthat each group will generally include at least the followingattributes.

A publisher may be an operator of a content delivery system or networkin which a user experience that is anticipated to primarily be a visual,audio, or audio-visual experience may be delivered. Publishers mayinclude website operators, application developers, or other entitiesthat deliver content to users of computers or other audio-visualdevices. In the inventive system, a publisher may set the maximum numberof user experiences (e.g., inquiries, questions, advertisements, videos,music, etc.) that may be delivered to a user through the inventivesystem. A publisher may also control the maximum number of questionsasked to a particular user within a particular time period and themaximum number of times that a particular user may convert on a specificcampaign provided by an advertiser. Importantly, a publisher provides acontent delivery platform (e.g., a website or application) into which auser experience provided by the inventive system and method may beplaced.

An advertiser may be an entity providing a user experience such as abanner advertisement, advertising video, advertising campaign, etc.Advertisements may include, for example, banner advertisements, leadgeneration, video, audio, cookie placements, mobile-basedadvertisements, advertisements for devices without screens (such ascertain smart home devices), etc. An advertiser may desire to collectparticular information from a user. An advertiser may specify that auser experience can be provided to particular user only afterdetermining that the user has certain characteristics or after the userhas indicated particular responses to posed questions. An advertiser mayspecify sets of attributes for which the advertiser determines thatthere is a particular utility in providing user experiences to userswith the attributes. In such instances, the advertiser may provide a bidin which the advertiser agrees to provide or transfer the identifiedutility to an implementer of the inventive system and method in exchangefor exposing the user to the advertiser's user experience.

The entity implementing the inventive system and method may be anadvertiser or publisher, but preferably is in a different category andis able to act as an intermediary between advertiser and publisher.Preferably this entity will operate a specialized computer server systemthat provides the functionality identified herein and which interfacesor communicates with the computer systems of the publisher, advertiser,and user.

A user may be an entity that preferably seeks a particular type ofcontent delivery from a publisher and to which a user experience isprovided as part of the delivery of the desired content.

In a simplified explanation of a portion of one embodiment of theinvention, an advertiser may want to indicate that the advertiserdesires to deliver its visual and/or audio advertising to users whoprefer the color purple. This indication may take the form of a bidindicating that delivering such advertising has a set amount of utilityto the advertiser and that the advertiser will surrender such amount ofutility to the party who delivers the advertising to the user. (In otherinstances, the advertiser may indicate that utility is derived from actsother than delivering advertising, for example, receiving a user inputindicating interest or receiving an indication that the user hascompleted a goal set by the advertiser.) The inventive system maycapture the amount of such utility, reduce such utility by theconversion rate of the campaign in which the advertising exists, reducethe resulting amount by the answer rate for a question designed toelicit the preference for the color purple (wherein the answer rate isthe likelihood that a user may answer a question in a manner in which itis possible to determine that the user prefers the color purple), reducethe resulting number again by a percentage indicating the manner inwhich the party implementing the inventive system and a party displayingthe advertising to the user may divide the received utility where thepercentage may vary based on whether the advertiser has a preexistingrelationship with the publisher, and thereafter reducing the resultingamount by a drop-off rate.

In a system wherein multiple types of user experiences are offered,multiple drop-off rates may be encountered. One exemplary userexperience may be a simple opt-in advertising campaign wherein the usermay be presented, for example, with an advertisement posing a yes/noquestion, such as “would you like to save 15% on car insurance?” In asimple opt-in situation, opting-in to that campaign will result in arecorded campaign conversion but will not result in directing the useraway from the user experience. Rather, the user could then progress tothe next experience in sequence. Users presented with this type of userexperience typically have a low drop-off rate. In the alternative, auser experience may be a banner advertisement that directs the user to adifferent website or otherwise directs the user away from the userexperience when selected by the user. Such user experiences have a veryhigh drop-off rate. The expected utility to be gained by providing suchan experience is reduced based upon this drop-off rate.

For example, a user may be presented with many types of user experience.These user experiences may include inquiries (such as a yes/no, multiplechoice, or other form of question), a banner advertisement, a videoadvertisement, etc. When presented with a simple opt-in campaign,statistics show that a user is often likely to answer the question at ahigher rate and continue through the user experience at a higher ratethan when a user is presented with a banner advertisement which mayderail the user from the user experience by sending the user into adifferent experience after clicking on a banner advertisement.Accordingly, banner advertisements typically have higher drop-off ratesthan yes/no questions.

While each type of campaign has a drop-off rate range, many are lowerthan the drop-off rates for banner advertising. Thus, based on thecalculations performed to determine the expected utility from providinga particular user experience, it is very unlikely that a banneradvertisement will be shown in the early portions of the userexperience. Achieving this result may be possible by seeding a banneradvertisement with a high drop-off rate before it has been presentedenough times for the inventive system to have calculated its actualdrop-off rate. For example, if the number of permitted inquiries is ten(10), and there is a new banner advertisement available in the inventivesystem, the inventive system may seed a high drop-off rate into thecalculation of expected utility for that banner advertisement for thefirst eight (8) inquiries so that it is unlikely that the banneradvertisement would be shown in the first eight (8) inquiries. Once acampaign has reached a volume of activity that allows the inventivesystem to calculate an accurate drop-off rate, the system will use thecalculated metric and not the seeded value anymore. In systems that donot employ drop-off rate, it is likely that a user experience with ahigh drop-off rate may be presented in an initial inquiry due to itshigh utility. But doing so will often reduce the overall utility to begained through the method because a user is likely to drop-off whenpresented with such an experience. Thus, it is preferable to employ adrop-off rate that will account for the desire to avoid users droppingoff of the user experience. Such a rate may be a pre-set default for newcampaigns, may be based on drop-off statistics, may be optimized toincrease utility to the publisher, or may be determined in anothermanner. It is preferable, for advertising objects that have not hadsignificant volume, to initially use the object with a pre-set defaultrate and record drop-off statistics for the object. After an object hasbeen used in a desirable number of instances, a more accurate drop-offrate may be statistically established. In such instances, if thedrop-off rate for a banner advertisement was sufficiently low, such anadvertisement may be shown as one of the early inquiries.

In certain embodiments of the invention, it may be desirable to performa deeper analysis based upon drop-off rate before providing further userexperiences. For example, a banner advertisement or other type ofcampaign that links-out of the user experience may present the highestutility at a particular point in the sequence of inquiries, yet have ahigh drop-off rate. In such instances, it may be desirable to perform adeeper drop-off-related analysis before presenting the user experienceto the user. Such analysis can include the system considering the numberof inquiries permitted minus the number of inquiries already used. Thenbased upon the remaining number inquiries permitted, the system maydetermine either all or a subset of potential campaigns that the usermay qualify for within the remaining number of inquiries. The system maythen perform an analysis of the answer rate for the questions or otherexperiences that are needed to gain the attributes required for thesecampaigns. The system may also consider the conversion rate of thesecampaigns. Based on this, the system may determine the estimated utilityof the remainder of the user experience based on the campaigns that maybe presented to the user throughout all or some remaining portion of theuser experience. The system may compare that expected utility to theutility of the campaign that has a high drop-off rate, in an effort todetermine whether the utility of the single campaign having the highdrop-off rate is greater than the expected utility of other campaignsfor which the user qualifies. A decision may be made based on thecomparison, and the user may be presented with the experience thatrepresents the higher utility. This analysis may be performed at eachsuccessive step of the user experience in an attempt to obtain a higheroverall utility related to the user's visit.

Answer rate considers the likelihood of a particular answer to aquestion. It may be desirable to alter the questions that are used sothat the quality of the answer is more important than the number ofclicks. In one example of using answer rate, if 12% of a population ofusers has diabetes, the likelihood that a user would answer a questionin a manner confirming that the user has diabetes is expected to be 12%.The actual answer rate will be calculated based on users' answers tothat inquiry. For inquiries designed to elicit one of multiple answers,each answer may have its own answer rate. For example, in a questionhaving answers A, B and C, the answer rates may be 70% for answer A, 20%for answer B, and 10% for answer C. The inventive system and method maytake such answer rates into account in determining the user experience.Similarly, if different user experiences present alternative ways ofgathering the same information, e.g., different inquiries, each of thealternative experiences may have different answer rates that get to thesame answer. In such cases, the inventive method and system can factorin the alternative answer rates in determining the appropriate userexperience to present. If a particular question or inquiry has not beenpresented to users often enough to determine an actual or projectedanswer rate, the system may be seeded with an initial answer rate thatmay persist until sufficient data has been gathered to prepare an actualanswer rate.

In addition to the factors identified above, the inventive methodcontemplates the use of additional factors that may assist inappropriately determining the utility to be gained by presenting a userwith a particular user experience. For example, factors such as thenumber of times that a user experience may be presented to a user andthe number of times that a user can convert in a particular userexperience may also be taken into account.

With respect to dividing the received utility, an advertiser may providean amount of utility to the party implementing the inventive systemthrough a publisher. In such a case, the implementing party may retain afirst portion of the utility and provide a second portion of the utilityto the publisher. However, in the case where the advertiser has apre-existing relationship with the publisher, the implementing party mayretain a smaller share of the utility and provide a larger share of theutility to the publisher.

Such indications of utility may take any form indicating a measure ofpreferences for one set of goods and services versus another set ofgoods and services. Utility may take a financial form, indicating anindividual's price for an asset. This price may be measured in goods,commodities, money or currency, or other measures of value.

As indicated above, in certain embodiments of the invention, thefollowing factors may be used as metrics to determine the desirabilityof providing a particular inquiry or user experience:

-   -   a. Amount of utility bid by an advertiser    -   b. Answer rate for an inquiry wherein, for example, each answer        to each inquiry may have its own expected rate of being chosen        and the inventive system may use the likelihood of obtaining the        desired answer    -   c. Conversion rate for an advertising campaign or user        experience    -   d. Maximum permitted number of inquiries that may be presented        to a user minus the number of inquiries that have already been        presented to the user    -   e. Maximum number of times that a particular user experience        (e.g., an advertising campaign) may be presented to a user        within a particular set of constraints (e.g., period of time)        minus the number of times that the user experience has been        presented to the user within the current set of constraints    -   f. Maximum number of times that a user can convert on a        particular type of user experience within a set of constraints        (e.g., a website visit) minus the number of times that the user        has converted on that type of user experience during the current        set of constraints    -   g. A split of utility between the publisher and the party        implementing the inventive method    -   h. Drop-off rate for various user experiences, e.g., the        probability that a user will disengage from a particular user        experience after being presented with the experience (for        example, how often a user will drop-off after being presented        with a particular campaign)

Data may be collected from some or all user visits to enhance theaccuracy of one or more of the metrics identified above.

Using only a portion of the metrics specified above may lead to adifferent and less desirable outcome. Consider an example in which WallyWorld owns a theme park and is a publisher that operates a websiteoffering free online games to users. In the example, Wally World maypermit an implementer to implement the inventive system and methods suchthat after a user registers for a membership, the implementer ispermitted to provide a user experience to the user. In the example, twocampaigns are bidding to be presented as part of the user experience.These exemplary campaigns are identified as “Go Go Car Insurance” (or GoGo) and “Wally World Theme Park.” (or WWTP). Assume that Go Go bids $10for providing a user experience and WWTP bids $8 for providing a userexperience. If only the bid amount is considered, the implementer wouldalways provide the Go Go user experience to the user via the publisherdue to the higher bid for the Go Go experience.

If the example is modified to indicate that both are advertisingcampaigns seeking a conversion and the campaign conversion rate is 30%for Go Go and 50% for WWTP, then the expected utility of providing theuser experience to the user is the bid times the conversion rate. Thisis $10*30% (i.e., $3) for Go Go and $8*50% (i.e., $4) for WWTP. In thisexample, the implementer would provide the WWTP user experience to theuser via the publisher.

If the example is further modified to require an inquiry before the userexperience as follows, the result may change. Assume that Go Go requiresan affirmative answer to the question “do you own a car?” and that suchaffirmative answers occur at a rate of 40%. Also assume that WWTPrequires an affirmative answer to the question “do you like themeparks?” and that such affirmative answers occur at a rate of 25%. Inthis instance, the answer rate can be factored into the utilitycalculation leading to an expected utility of $1.20 for Go Go (i.e.,$10*30%*40%) and an expected utility of $1 for WWTP (i.e., $8*50%*25%).

Assume that the example is further modified such that the amount ofrevenue that the implementer shares with the publisher differs for eachadvertiser. For example, if 72% of revenue from Go Go is shared with thepublisher and 95% of revenue from WWTP is shared with the publisher,then Go Go presents an expected utility of $0.86 (i.e., $10*30%*40%*72%)and WWTP presents an expected utility of $0.95 (i.e., $8*50%*25%*0.95).In this instance, it is more desirable to the publisher to present theWWTP user experience than the Go Go user experience.

Assume that the example is further modified such that the drop-off ratefor the Go Go user experience is 10% and the drop-off rate for the WWTPuser experience is 50%. In such an instance, Go Go would present ahigher expected utility to the publisher, because the publisher wouldexpect to capture 90% of the $0.86 from Go Go but only 50% of the $0.95from WWTP.

Thus, it can be seen that the specific set of metrics used to determinethe user experience to present may result in differing results even inthe simplistic example presented above. In a real-world system havingtens, hundreds, or thousands of user experiences to be presented, thevariance may be more pronounced.

Many prior art systems present users with a predetermined userexperience that either presents each user with the same experienceregardless of information gathered from the user during the user'sprogression through the experience and without attempting to determinewhether the progression should be altered. Other prior art systems mayhave a tree that branches, for example, if a user answers “yes” to ayes/no inquiry, a first branch is taken and if the user answer “no” tothat question a second branch is taken. However, as with the other priorart systems, such branching systems do not dynamically attempt todetermine the user experience to enhance the utility gained in theprocess. The inventive system, however, overcomes these predeterminedsingle path or branching path systems by providing for a dynamicallydetermined user experience and may implement this dynamic experience inthe manner described herein to enhance the functionality of thecomputing devices used in the inventive system and method.

In an embodiment of the inventive system and method, at each step of theprocess after a user has completed a portion of the user experience, thesystem may take one of three steps based on the information that isavailable to the system. It may (1) present the user with an additionalinquiry, (2) present the user with another type of experience (e.g., anadvertising campaign), or (3) end the user experience. This ispreferentially dynamically determined according to the methods set forthherein.

Referring now to FIGS. 1A and 1B, the inventive system and method may beimplemented using a services-oriented architecture, micro servicesarchitecture, or event based architecture, rather than a genericcomputer architecture to enhance the speed and efficiency over whatwould be encountered in the prior art systems. This enhancement allowsscalability to large numbers of users and reduces cycle times tohundreds of microseconds, rather than the multiple seconds encounteredin prior art systems and methods. The architecture implements multipleservices to handle large volumes of data asynchronously as the dataflows through the system. In FIGS. 1A and 1B, nodes a, b, c, d, e, f, g,and h indicate connections between the two FIGS.

System 100 may comprise multiple services 110, 120, 130, 140, 150, 160,170, and 180.

Front end service 110 provides a user experience to a user, preferablythrough a publisher website. Service 110 is the service by which anexperience may be presented to a user and by which user input may beobtained. It is preferable to direct the user experience data to apublisher and permit the publisher to use the user experience data toprovide the user experience to the user. However, it is also possible topresent the user experience directly to the user. Publishers may usewebsites, applications, mobile device applications, and other methods topresent user experiences to users. When a publisher indicates (or a userdirectly indicates) that a user experience should begin, process 112 isinitiated and a notification 114 is posted by process 112 to the process122 of the decision service. Process 112 remains in place to receivefeedback notification 128 from process 122 of the decision service 120.Continuation arrow 116 indicates that process 112 may continue beyondnotification 128 and send one or more additional notifications 114 todecision service 120 as a user progresses through the user experience.

Upon receipt of notification 114, process 122 of decision service 120begins requesting information from and tracks the returned informationfrom many of the services. Decision service 120 may be used to overseemuch of the process regarding the appropriate inquiry, e.g., question oradvertising campaign, to eventually transmit to front end 110 vianotification 128. Continuation arrow 129 indicates that process 122 maycontinue to operate beyond what is indicated in this figure.

Process 132 of visit tracking service 130 receives a request for atracking ID from decision service 120 via notification 124. Visittracking service processes the request and returns a visit object thatmay include a tracking ID to process 122 via notification 134.Continuation arrow 136 indicates that process 132 may keep operatingbeyond what is indicated on this figure.

Campaign filtering service 140 may be used to match users to appropriateuser experiences (e.g., advertising campaigns) based on user attributes.For example, data indicating gender, location, and type of user devicemay be used by service 140 to filter various types of user experiences.Service 140 may also consider other user specific attributes such aswhether a user owns a dog, whether a user is seeking insurance, etc. tomatch users to appropriate user experiences and/or filter outinappropriate user experiences. Process 142 of campaign filteringservice 140 may receive a request to match campaigns from process 122via notification 125. Such a request may a visit tracking ID and othervisit object information included. Eventually, after communications withservices 150 and 160, process 142 may return a filtered list ofcampaigns to process 122 via notification 149. Continuation arrow 148indicates that process 142 may continue beyond what is indicated on thisfigure.

Campaign availability service 150 may be used to determine whether aparticular user experience or advertising campaign may be available. Forexample, if a campaign is limited by an advertiser to certain times ofday (e.g., before 6:00 pm) or limited in the number of times that it maybe delivered to users based on the overall utility to an advertiser ofdelivering a campaign, the availability service 150 may determinewhether such criteria have been met. Service 150 may be augmented by abudgeting service when a budget is specified by an advertiser. Process152 of service 150 may receive a request for available campaigns vianotification 144 from process 142. Notification 144 may include a listof appropriate campaigns for which process 142 is requesting anindication of availability or unavailability. Process 152 is thenconfigured to determine the availability of the various identifiedcampaigns based on the appropriate availability criteria and return alist of such campaigns to process 142 via notification 154. Such listmay comprise a list of campaign IDs. For efficiency, it may bepreferable to provide a database to service 150 wherein such databasemay be limited to storing campaign IDs and availability criteria.Continuation arrow 156 indicates that process 152 may continue beyondwhat is indicated on this figure.

Campaign administration service 160 is a service that allows advertisersor their agents to provide information used for administering campaignswithin the inventive system. Such information may include identifyinginformation, utility bids or indications, and other relevant informationas is discussed in additional detail below. Process 142 may provide alist of available campaign IDs to process 162 of service 160 by way ofnotification 146. Such notification may be an indication that relevantinformation is needed regarding the identified campaigns. Afterretrieving the relevant information, process 162 may return the list ofcampaigns and the relevant information to process 142 via notification164. Continuation arrow 166 indicates that process 162 may continuebeyond what is indicated on this figure. As noted above, process 142 maythen provide further filtering of the list of campaigns before returninga filtered list of campaigns via notification 149 to process 122. Thisfiltering may be based on personally identifiable information (PII) suchas name, age, address, gender, email domain name, device type, etc. Itmay also consider campaign filtering attributes, such as campaign agerange, campaign desired gender, location, etc.

Information stored in the campaign administration service 160 databasemay include images, sounds, videos, information related to advertisingcampaigns, advertising materials, and information that provides links tosuch materials if such materials are stored using a remote service suchas Amazon's S3 service or a similar cloud storage platform.

After receipt of a filtered list of campaigns via notification 149,process 122 may request via notification 126 that process 172 ofattribute value calculation service 170 determine all question (orinquiry) series that have a cost less than or equal to the remainingnumber of questions (or inquiries) allowed. This request is intended todetermine which relevant series of questions may be permitted within theconstraints related to the number of inquiries permitted by thefront-end service. For example, it may be desirable to determine whethera user owns a 4-wheel drive vehicle. Doing so may require asking twoquestions: 1) “do you own a vehicle?”; and 2) “is your vehicle a 4-wheeldrive vehicle?” Such questions may be desirable for an automobileinsurance company, specialty magazine, or other advertiser who wishes toadvertise or collect information from owners of 4-wheel drive vehicles.In this example, the question series is the indicated list of questionsneeded to obtain the ultimate attribute(s) needed for a particularcampaign or user experience. The exemplary question series included twoquestions, however, question series may include any number of questions.Question series are preferably generated by determining an attribute gapthat indicates attributes needed for a particular campaign that are notalready known to the advertiser or service providing the inventivemethod. After determining this gap, it is preferably to generate aseries of questions that, if answered, resolve the gap by determiningwhether the needed attributes are present. In the instance where anattribute is needed by more than one campaign, it may be possible toshare question series or portions of question series to reduce the costof identifying any particular attribute. For example, if one campaigndesires to know whether a user owns a purple vehicle and anothercampaign desires to know whether a user owns a 4-wheel drive vehicle,both might have a common first attribute that may lead to a common firstquestion (e.g., “do you own a vehicle?”) and differing second attributesthat may lead to different second questions.

Thus, the attribute value calculation service 170 considers theremaining number of inquiries permitted by the publisher and determinesall relevant question series that include a number of questions that isequal to or less than the remaining number of inquiries permitted.Process 172 may return all of the appropriate question series to process122 via notification 174. Continuation arrow 176 indicates that process172 may continue beyond what is indicated in this figure.

The data model training service 180 implements process 182 to calculatethe campaigns that provide the most utility based upon the filteringthat has occurred prior to invoking service 180. Upon receiving arequest for a relevant trained data model from process 122 vianotification 127, process 182 will preferably multiply the amount ofutility that was bid in a campaign by a conversion rate. The product ismultiplied by the answer rate, which results in a product that ismultiplied by the percentage of utility provided to the operator of thesystem. That product is multiplied by the drop-off rate, as indicatedabove. This is preferably performed for each of the identified campaignsand results, for each campaign, in an expected utility to be gained bypresenting each of the possible user experiences to the user. After eachof these expected utility values is determined, process 182 returns theexpected utility values to process 122 via notification 184.Continuation arrow 186 indicates that process 182 may continue beyondwhat is indicated in this figure.

Upon receipt of notification 184, process 122 may determine the datamodel with the highest expected utility and use that data model todetermine the contents of notification 128, which is then sent toprocess 112. In certain instances, the relevant trained data model(s)may indicate that it is not desirable to provide a further userexperience. In such an instance, notification 128 may indicate toprocess 112 that no further user experiences are available. For example,if a publisher only permits user experiences for which the publisherobtains a minimum threshold amount of utility and notification 184 doesnot return any data models that meet the minimum threshold, thennotification 128 may indicate to process 112 that no further userexperiences will be provided. In the instance that no further userexperiences are provided, front end service 110 preferably indicates tothe publisher that the user should be directed to content provided bythe publisher.

The process illustrated in FIGS. 1A and 1B may be repeated multipletimes. For example, in a user visit in which the publisher allows 10inquiries of the user, the process outlined in FIGS. 1A and 1B may berepeated 10 times to provide 10 user experiences to the user. Repeatingthe entire process may be necessary because certain filteringattributes, budget attributes, or other restrictions may have occurredin the time since notification 114 was initially sent. For example, if auser begins the process at 5:59 pm and takes a few minutes with the userexperience returned in notification 128, then any campaign that isunavailable after 6:00 pm will be unavailable when the user is preparedto request or trigger the next user experience. Thus, the process ofFIGS. 1A and 1B should preferably be repeated to ensure that thenow-unavailable campaign is not included in the results. Other changesin circumstance may also require this, including for example if anadvertiser disables a campaign, if a campaign budget cap is met, or if acampaign or question reached its default metric threshold and thedecision engine switched over to using the actual metrics for thatobject.

In some embodiments it may be desirable to split one or more of theabove-identified services into multiple services if the load on aparticular service is too great. It may also be desirable to removecertain services or to add additional services if additional featuresare added to the system.

Referring now to FIG. 2 , a representation 200 of data flow within theinventive system and method is provided.

Visit event tracking service 210 corresponds to the data flow that isexpected to occur after notification 128 returns a campaign or questionto process 112 and the user is given an impression of the campaign orquestion. When a question is provided to a user, QuestionImpressed event211 occurs. In connection with this impression of the question, a usermay either answer a question (in which case QuestionAnswered event 212occurs) or skip the question (in which case QuestionSkipped event 213occurs). In the instance where any of events 211, 212, or 213 occur,each of those events that occurs is picked from the data stream by visittracking service 260. Service 260 then modifies two database documents,i.e., visits document 270 and visitors document 280, to indicate theresults of the relevant events of events 211, 212, and 213. It may bepreferable in some embodiments to provide a database using JSON-likedocuments with schemas. One such database is identified using thetradename mongoDB and is provided as an open source software project.

Because FIG. 2 illustrates data flow among asynchronous services, itshould be noted that certain events are identified as “Raw” events. Suchdesignation is useful because, in certain embodiments, multiple servicesmay be pulling the same items from the data stream and performingoperations related to such items without regard to whether otherservices are also using the same items. Such designation may also beuseful if the user causes duplication by refreshing the browser,clicking an item more than once, using a browser that is havingproblems, etc. Accordingly, duplication of events or results of eventsmay occur if proper controls are not implemented. By way of example, ifa user views a campaign, the system may insert a CampaignImpressed eventinto the data stream. At that point, in certain embodiments, it may bepossible that more than one of the services that is viewing the datastream may act upon the CampaignImpressed event when each of theseservices identifies that event in the data stream. These services mayinclude services that track budget for a campaign, services that trackconversion rates for a campaign, etc. Each service may be altering orappending data in the CampaignImpressed event simultaneously or inoverlapping fashion. If and when this happens, the services may bothreinsert the CampaignImpressed event into the data stream with slightlydifferent data, such that the single CampaignImpressed event appears tobe two separate events in the data stream. In other embodiments,duplication may occur due to user error or other functionality problems.Such duplication can be prevented by providing a unique identifier foreach event and implementing a deduplication service, such as EventDeduplication service 230 to eliminate duplication.

To prevent certain instances of duplication, it is preferable to use aslightly different event that will be handled only by the deduplicationservice 230 when that event occurs in the data stream. In this exemplaryembodiment, CampaignImpressedRaw event 201, CampaignClickedRaw event202, and CampaignSkippedRaw event 203 respectively indicate whether auser was given an impression of a campaign and either clicked on thecampaign or skipped the campaign. Appending the “Raw” tag to the event,in this example, is a signal that only the deduplication service 230should handle the event. After the deduplication service pulls one ofthe “Raw” events from the data stream, service 230 inserts acorresponding event or events into the data stream from the set ofCampaignRecommended event 204, CampaignImpressed event 205,CampaignSkipped event 206, CampaignClicked event 207, andCampaignConverted event 208. Visit Tracking service 260 will pick upeach of these events from the data stream as indicated by the arrowsfrom each of events 204 to 208 that extend to service 260.

In addition, Conversion Tracker service 240 will pick upCampaignImpressed event 205 and CampaignClicked event 207 from the datastream. The CampaignRecommended event 204 is also picked up by PixelTracking Service 250, which inserts a PixelFired event 209 into the datastream. The PixelFired event 209 is picked up from the data stream byConversion Tracker Service 240.

A Conversion Tracker Service 240 such as this is useful where the systemis intended to support multiple manners of permitting conversiontracking. One such manner of tracking conversions is a “CPM” (or costper thousand impressions) model wherein an item having utility istransferred in exchange for every thousand impressions of the campaign.Thus, for a CPM model, it is important that CampaignImpressed event 205be captured by service 240. Another manner is a “CPC” (or cost perclick) model wherein an item having utility is transferred for everyclick. Thus, for a CPC model, it is important that CampaignClicked event207 is captured by service 240. And a third manner is a “CPA” (or costper action/acquisition) model wherein an item having utility istransferred based on a particular action or acquisition, e.g., anindication that a user signed up for a new service, purchased a product,etc. Thus, for a CPA model, it is important that PixelFired event 209 becaptured by service 240. Each of these events captured by service 240 ispreferably used to document when an item having utility should betransferred from an advertiser to another party. After ConversionTracker Service 240 receives one of events 205, 207, or 209, service 240generates an appropriate CampaignConvertedRaw event 216 in the datastream. Preferably, only event deduplication service 230 will handleevent 216 and thereafter, when it is not a duplicate, service 230 willinsert a CampaignConverted event 208 into the data stream for processingby Visit Tracking service 260.

When a CPA campaign is used within the inventive system, an advertiseris provided with a unique pixel address that may be provided to a userviewing the campaign. This pixel may be provided by the entityimplementing the inventive system. The pixel may be assigned a uniquepixel address so that the entity may record information about the pixelbeing accessed. The pixel may be provided to the user by the entityproviding code to an advertiser; the advertiser may then insert thepixel and code into the advertiser's website (e.g., on a conversion pageor a goal-met page) or within an email indicating that a goal has beenmet. Such goals may include a user signing-up, a sale of a product, areferral sent, etc. Upon the user viewing the campaign, e.g., on awebsite, the pixel address is loaded by the user's system, which causesan indication within the data stream that the pixel was loaded. Thisindication will result in the PixelFired event 209 being inserted intothe data stream. It is also possible to send a data payload to theentity when the pixel is loaded; such data may be used to track aspectsof the user's visit to the website or other browsing information.

It is important to note that for each unique user event or action, it islikely that only a single thread will flow through the data stream,although that thread may split. For example, if a user clicks upon anadvertising campaign, the CampaignClickedRaw event 202 will be generatedand placed into the data stream by Visit Event Tracking service 210.That event 202 will be processed for deduplication by service 230 and,upon deduplication, service 230 will place a CampaignClicked event 207into the data stream. The CampaignClicked event 207 will be processed byVisit Tracking service 260 and, if it is a part of a CPC model, it willalso be processed by Conversion Tracker service 240. Conversion Trackerservice 240 will generate a CampaignConvertedRaw event 216 and place itinto the data stream. Service 230 will act upon event 216 to deduplicateit. Upon deduplication, if the event 216 was not a duplicate, thenservice 230 will insert a CampaignConverted event 208 into the datastream. Service 260 will pick up event 208 from the data stream and makeappropriate amendments to either or both of documents 270 and 280. In asimilar manner, other types of user or system actions may result in dataflowing through the data stream and relevant services, as indicated bythe flow arrows in FIG. 2 .

Due to the potentially high volume of data in the data stream, it ispreferable to use a streaming platform that can accommodate high volumesof data from many users simultaneously. One system that may be used as astreaming platform is marketed under the tradename Kafka by the ApacheSoftware Foundation. Acceptable systems will preferably allow a numberof services to view and operate on the data stream simultaneously andasynchronously. When using a service such as this, each service operatesindependently and seeks the data that it is configured to seek in thedata stream. This allows for an easily changeable or scalable systembecause services can be added or removed without directly affectingother services. In a preferred embodiment of the invention, a streamingplatform may be operated on one or more servers devoted to optimizationof the streaming platform while the various services are operated on oneor more separate servers. Each service may be operated on its ownserver, a parallel arrangement of multiple servers, or on a servershared with another service. However, it is preferred that the streamingplatform be operated on its own server(s) due to the expected highvolume of data-handling expected in such platform.

In the exemplary data flow shown in FIG. 2 , it should be noted thatnone of events 211, 212, and 213 is processed by the deduplicationservice 230. However, this is depicted in this manner to indicate thatif a limited set of services is operating on a set of events such asevents 211, 212, and 213, deduplication may not be necessary.Additionally, certain services may not need deduplication because useror browser activity is less likely to cause relevant duplication.However, it should be understood that where necessary due to processingrequirements or desires, the deduplication service can be configured toact upon additional events including, for example, events 211, 212,and/or 213. Similarly, the deduplication service 230 can be configuredto cease acting on other events.

Decision service 220 corresponds to decision service 120. Similarly,notification 128 which returns either a campaign or question correspondsto the events emerging from service 220. When a question is returned innotification 128, this corresponds to decision service 220 inserting aQuestionRecommended event 215 into the data stream. Similarly, whennotification 128 returns a campaign, this corresponds to decisionservice 220 inserting a CampaignRecommendedRaw event 214 into the datastream. In the inventive system, after decision service 220 recommends aquestion or campaign, a user may be impressed with the question orcampaign resulting in Visit Event Tracking service 210 generating eithera CampaignImpressedRaw event 201 or a QuestionImpressed event 211. Insome instances a user system failure or user action may result in thecreation of an event 214 or 215 without the corresponding creation of anevent 201 or 211. This may occur when an interruption of service,computer failure, browser closing, or other event occurs after the firstevent and before the second event. Accordingly, it is preferable toinsert and track these types of events separately.

It is preferable, in the data flow depicted in FIG. 2 , to use a visitsdocument 270 to track the events that occur in a single visit by asingle user to the service. Similarly, it is preferable to use avisitors document 280 to track relevant data related to all of thevisits by a particular visitor to the service. This tracking in document280 may be aggregated over time or limited to a certain time period.Visits may be tracked, for example, by a combination of PII or otherrelevant information related to a visitor, such as name, email address,IP address, phone number, cookie on a browser, or other data that theuser may provide to the service.

Referring to FIG. 3 , a suppression and verification feature data flowis depicted. The suppression data flow may be used to determine whetheran advertiser has a particular customer or user within their databaseprior to impressing the user with an advertising campaign. For example,if an advertiser has a user named “Bob Smith” as a client, thatadvertiser may indicate that it does not desire to pay for deliveringits advertising campaign to user “Bob Smith.” Thus, it is desirable tobe able to query whether the advertiser recognizes the user prior todelivering a campaign to the user.

Unlike known services, the inventive service implements a novelsuppression method. Existing systems may employ a suppression methodwherein a suppression check or “pre-ping” is run prior to delivering anadvertisement or portion of a campaign. In existing systems, if thesuppression check returns an error or does not return a timely result,the systems may impress a user with an advertisement or campaign, butlater learn that the advertisement or campaign should have beensuppressed for the user. This results in loss of revenue because anadvertiser often will not pay to deliver an advertisement to or toconvert an existing customer.

In the inventive system, if the advertiser does not respond to asuppression check affirmatively indicating that it is permissible todeliver an advertisement, then the advertisement will not be delivered.This method goes against conventional industry knowledge, but results inless revenue lost due to delivering advertisements for which anadvertiser will not pay. This method is further enabled by the processset forth in FIGS. 1A and 1B. Specifically, the campaign availabilitycheck performed through notification 144 and process 152 may beintegrated with the suppression check such that if an advertiserresponds that a user need not be suppressed, process 152 may be promptedto include that indication as part of its determination of whether thecampaign is available. In the alternative, if an advertiser does notrespond or responds that a user should be suppressed, then process 152may be prompted to exclude the related campaign(s) from the list ofavailable campaigns. It is possible, however, that in a later iterationof the FIGS. 1A and 1B process, the suppression result may change and acampaign may be included in the available campaign list. For example, ifthe advertiser delays its response to the suppression request for 2seconds, the first iteration of process 152 may not include thatadvertiser's campaign(s) in the list of available campaigns. However, ifthe user continues to interact with the service, a later iteration ofthe FIGS. 1A and 1B process that occurs after that 2 second mark mayinclude the previously excluded campaign(s).

Using the inventive suppression method reduces waste of resources byproviding users with a user experience that is meaningful andpotentially new, rather that provide users with already knowninformation or information related to a company of which the user isalready a client. This inventive method further reduces burden onadvertisers who no longer need to implement processes to confirm whethertheir content was delivered to known users and reduce bills based on anyinstances of such. The inventive method further enhances the utility ofthe system to other advertisers who may have a lower bid, because ratherthan being excluded due to an unsuppressed user, in the inventivesystem, the user has a higher likelihood of seeing thelower-bidding-advertiser's content.

As illustrated in FIG. 3 , Visit Service 330 creates a VisitCreatedevent 301 and inserts it in the data stream. Visitor PII Verificationservice 310 processes event 301, as does Suppression service 320.Service 310 may use various means to determine whether the user hasprovided false or faulty identifying information, including name, emailaddress, phone number, mailing address, etc. Service 310 may be aninternal service or may refer the verification to an outside company andsend that company the necessary information for verification, such asphone number, email address, name, etc. Upon completion of theverification process, including receipt of any information from anoutside vendor, service 310 places a VerificationProcessed event 315into the data stream. This event 315 is processed by Visit Service 330.

Suppression service 320 also processes event 301 from the data stream.As noted above, service 320 sends requests to advertisers to verifywhether the user is a user for whom advertising should be suppressed. Itis possible that event 301 sends such a request to every advertiserusing the service, which could be tens, hundreds, or thousands ofadvertisers. In the alternative, advertisers may trust the service tohandle suppression and may provide a suppression database so that theservice 320 may send the request for verification to an internaldatabase. Upon receipt of suppression information, service 320 places aSuppressionProcessed event 325 into the data stream. Because service 320may receive data from hundreds of advertisers, it may be necessary forservice 320 to place hundreds of events 325 into the data stream, aseach request advertiser sends the suppression information to service320. Alternatively, it may be possible for the suppression data to beprocessed by allowing access to a suppression database or allowing anupload of data in a batch or file prior to suppression processing.Ideally, the suppression check should be performed in hundreds ofmilliseconds such that the system can complete the entire process inless than a second. It may be preferable to pause the processing of theuser's visit for 500 milliseconds to permit PII verification andsuppression by services 310 and 320. It is also desirable to handlesuppression through direct access to a third party database wherein theadvertiser informs the service of the manner in which suppressionservice 320 can directly query and receive a response from theadvertiser's suppression database to ensure that the request isprocessed promptly. If an advertiser does not offer a suppressionservice, it is desirable to provide an indication that the advertiser'scampaign is deemed to be not suppressed. Finally, it is desirable toexecute the suppression process only a single time for a user within arelevant time period, e.g., the suppression process might only beexecuted once per day for a given user. Event 325 is processed by VisitService 330.

Visit Service 330 updates visits document 270 and visitors document 280with the results obtained from events 315 and 325. Service 330 maycorrespond to service 260.

Referring to FIG. 4 , this FIG illustrates an exemplary process flow forprocesses in which account administration or campaign administrationdetails are modified. An advertiser may utilize Account Admin service410 to modify account information as set forth in further detail below.Upon modification of account information, service 410 places anAccountChanged event 415 into the data stream. Similarly, an advertisermay use Campaign Admin service 460 to modify information related to acampaign as set forth in further detail below. Upon modification ofcampaign information, service 460 places a CampaignChanged event 465into the data stream. Question Series Service 430 and AvailabilityService 420 both process event 465 from the data stream. And service 420also processes event 415 from the data stream. AvailableCampaignsdocument 425 may be updated by Availability Service 420. Similarly,QuestionSeries document 435 may be updated by Question Series Service430. Although documents 425 and 435 may be implemented using anycompatible database, it is preferable to implement both types ofdocuments with the same type of database systems as is used to implementdocuments 270 and 280.

One example of a change to a campaign that might require a change to thequestion series would be a change by an advertiser who was previouslywilling to pay 1 unit of utility to determine whether a user possessed acar to a condition that the advertiser would later be willing to pay 100units of utility to determine whether a user possessed a 4-wheel drivevehicle. Such a change in service 460 would preferably trigger a changeto a questionSeries document 435 by service 430 to indicate that now twoquestions would be required rather than a single question.

An example account change might be a change to indicate that anavailable campaign was no longer available or vice versa. Such a changewould trigger an event 415 to be processed by service 420 that wouldresult in the modification of document 425. Another exemplary changemight be a modification to increase or decrease a budget or modify aschedule. Such changes might make a previously available campaignunavailable or might make a previously unavailable campaign immediatelyavailable. For example, if a campaign had reached a daily budget andbecome unavailable, then the advertiser increased the budget, such achange may result in the campaign becoming immediately available.Similarly, a change in the hours of campaign availability might renderan available campaign immediately unavailable or render an unavailablecampaign immediately available. It will be understood that other changesin an account or campaign may trigger an appropriate event 415 or 465and a corresponding change to one or more of documents 425 and 435.

Referring now to FIG. 5 , a representation 500 of data flow within theinventive system and method is provided. FIG. 5 refers to many of thesame elements and data flows that are represented in FIG. 2 . Asdepicted herein, Campaign Performance Aggregation service 520 acts onCampaignImpressed event 205 and CampaignConverted event 208 from thedata stream. Service 520 uses the data in events 205 and 208 to updatecampaignAggregationStates document 525, which itself may be implementedusing the same database system as is used with respect to documents 270and 280. Document 525 is preferably used to store the conversion rateidentified above that is used in determining the user experience thatshould be presented to users. A decision engine may read document 525 toretrieve the conversion rate for one or more relevant campaigns. Thenumber of campaign impressions recorded in events 205 and number ofcampaign conversions recorded in events 208 may be compared to determinea rate at which users who are impressed with the campaign actuallyconvert the campaign. The aggregated conversion rates and/or totals(rather than raw data) may be stored in document 525 to increase systemperformance and reduce the number of calculations needed to determinewhich user experience to provide to a user.

It is worth noting that a campaign will have conversion rate thatusually varies based on the placement of the campaign. If no conversionrate is known due to a small number of users who have seen the campaign,it is possible to use a default conversion rate until enough data hasbeen gathered to calculate a conversion rate with some level ofconfidence. For example, the inventive system and method may be given adefault conversion rate of 32.5% that should be used until 300 usershave been impressed with the campaign and then switch to the actualconversion rate after 300 users have been impressed. Similarly, theanswer rate for a given answer to a question may be given a defaultanswer rate until a sufficient number of users has answered a question.For example, a question with 4 answers may be assigned a default answerrate of 25% to each of the answers until at least 350 users haveanswered the question. The system may be instructed to use the actualanswer rate after 350 users have answered the question.

Also as depicted herein, Question Performance Aggregation service 510acts on QuestionImpressed event 211, QuestionAnswered event 212,QuestionSkipped event 213 and QuestionRecommended event 215 from thedata stream. Service 510 uses the data in events 211, 212, 213, and 215to update questionAggregationStates document 515, which itself may beimplemented using the same database system as is used with respect todocuments 270 and 280. Document 515 is preferably used to store theanswer rate identified above that is used in determining the userexperience that should be presented to users. A decision engine may readdocument 515 to retrieve the answer rate for one or more relevantquestions and/or answers of that question. The number of questionrecommendations and impressions recorded in events 211 and 215 and thetype and quantities of answers and/or skips recorded in events 212 and213 may be compared to determine a rate at which users who are impressedwith a question provide a particular answer. The aggregated answer andskip rates and/or totals (rather than raw data) may be stored indocument 515 to increase system performance and reduce the number ofcalculations needed to determine which user experience to provide to auser.

Referring to FIGS. 6A, 6B, and 6C, an exemplary layout for capturinginformation related to the users which an advertiser wishes to target isprovided to illustrate an embodiment of providing this functionality toan advertiser. A customize network block 610 may be provided to permitthe advertiser to select publishers' properties on which the advertiserwishes to make its user experience available. As illustrated, theexemplary advertiser has chosen Pete's Wardrobe and Daily Style INC. Anoption to manage properties may permit an advertiser to select multipleproperties from a list of existing properties. An advertiser may begiven the opportunity to bid different amounts for placements withineach property. Node w indicates the connection between FIGS. 6A and 6B.Node I indicates the connection between FIGS. 6B and 6C. Node xindicates the connection between portions of FIG. 6C.

A demographic block 620 may be provided to permit an advertiser tospecify demographic characteristics of a desired user group. Suchcharacteristics may include age ranges, gender, location, type of userdevice, email service provider, etc. Within some or all of thedemographic groups it may be desirable to provide an option to biddifferent amounts based on the demographic. For example, it may bedesirable to bid a higher amount for providing a user experience to afemale user than to a male user. Or it may be desirable to bid a higheramount to provide a user experience to a tablet computer user than to amobile device user. In some demographic categories, multiple choices maybe provided, for example, a listing of various device types on which theuser experience may be shared could include desktop, mobile, tablet,in-app, and/or other types of devices.

In another example, an advertiser may provide (a) a targeting list thatincludes identifying information for particular individuals, such asemail address, phone number, or other identifiers, and (b) a list ofsought attributes (for example, whether the individual expects to changecell phone service providers within a particular period of time, e.g.,within the next three months), and (c) a bid indicating that deliveringthe sought attributes has a set amount of utility to the advertiser andthat the advertiser will surrender such amount of utility to the partywho delivers the attributes to the advertiser. In such instances, thesystem may perform a check to determine whether the user is included inthe targeting list. If so, then the bid may be considered along withother bids when determining whether to present relevant questions orexperiences to the user in an attempt to determine the soughtattributes. If the bid is sufficient when considered in concert with theother relevant factors, then an experience may be presented to the userin an effort to determine the relevant attributes.

A visitor attribute block 630 may be provided to permit an advertiser todetermine specific attributes of a user to which the advertiser may wishto provide a user experience. Such attributes may include whether theuser has or relates to arthritis, auto insurance, no auto insurance,back pain, beauty, cooks, diabetes, dining out, financial investing,financial planning, fashion, shopping deals, etc.

Referring to FIGS. 7A and 7B, an exemplary layout for capturing bids andproviding statistics related to bids is provided to illustrate apreferred method of providing such functionality to an advertiser. A setof targeting parameters for desired users may be provided, such thatdiffering amounts may be paid for the same campaign depending on whetherthe user receiving the user experience has a set of characteristics thathas higher utility or lower utility to an advertiser. For example, inthe “Target” column, it can be seen that age, gender, type of device(e.g., mobile, laptop, desktop, etc.), and interests (e.g., shoppingdeals, fashion, etc.) can be set as parameters for a multiple bids for acampaign. Node y indicates the connection between FIGS. 7A and 7B.

As can be seen in the first two rows, an advertiser bidding on aplacement with publisher “Pete's Wardrobe” may value a female userhigher than a male user as indicated by the current bid of $2.50 for afemale user versus $0.50 for a male user. As seen in the first and thirdrows, a male interested in “fashion” may be valued higher than a similarmale interested in “shopping deals” as indicated by the current bid of$0.50 for a male interested in fashion as opposed to $0.20 for a maleinterested in shopping deals. Differing placements (as indicated in theplacement column) may affect the utility of the user to the advertiser,resulting in bids that differ based on the “placement.”

As illustrated in FIGS. 7A and 7B, a property column may be provided toindicate the publisher's website or app through which the campaign maybe published. A placement column may be provided to indicate differentoptions for where a campaign is published within a publisher's system. Acurrent status column may be provided to indicate whether a campaign isactive, paused, or enjoying another status. A current bid column andcurrent daily limit column may be provided to allow an advertiser toinput the utility value of bids and the maximum daily utility that acampaign may have to an advertiser. A cost column may be provided as anindicator of the cost of the campaign, which may be computed by summingthe cost of each billable lead (or conversion). An impressions column(“Impr”) may be provided as a count of how many times the campaign wasdisplayed to users. An Opt-In % column may be provided as an indicatorof how well a campaign is performing with users and can be calculated bydividing the number of “Billable Leads” by the number of “Impr”.“Billable Leads” (or conversions) may be presented as a count of thenumber of times users converted on a campaign. “Cost per Billable Lead”may be presented to indicate the average rate at which an advertiser ispaying for leads and can be calculated by dividing the “Cost” by the“Billable Leads”. An accepted lead percentage may be indicated in the“Accepted Lead %” column as calculated by dividing the number of“Accepted Leads” by the number of “Billable Leads”. “Accepted Leads” maybe provided as a count of the number of times an advertiser successfullyadded a lead to their database. The cost may be divided by the number ofaccepted leads to generate a cost per accepted lead as indicated in the“Cost per Accepted Lead” column. Finally, an eCPM value may bedetermined by dividing the cost by the number of impressions and thenmultiplying the result by 1,000 to generate an expected cost perthousand impressions. This figure may be displayed in the eCPM column.

In FIGS. 7A and 7B, all indicators of utility, denominated “bid,”“limit,” “cost,” or “eCPM” are denominated in dollars. However, one ofordinary skill will recognize that utility may be denominated in variousmanners and that dollars is merely as stand-in for purposes ofclarifying one potential use of the inventive system and method.

Referring to FIGS. 8A and 8B, an exemplary layout for capturingpublisher settings is provided to illustrate the type of data that maybe collected from publishers in certain embodiments of the invention aswell as a manner of collecting such data using browser software and amarkup language such as HTML. A settings block 810 may be used to name atype of user experience and a provide a description of the type of userexperience for future ease of reference. An experience block 820 may beused to identify the maximum number of inquiries that may be presentedto a particular user. It is also preferable, in experience block 820, topermit a publisher to identify whether users are permitted to skipcertain user experiences (e.g., questions) and to identify a minimumeffective CPM (“eCPM”) that the publisher will accept to allow acampaign to be published on the publisher's placement. Node z indicatesthe connection between FIGS. 8A and 8B.

Campaign type selections are preferably permitted in blocks 830 and 840.A campaign selection block 830 may permit a publisher to indicate thetypes of campaigns (e.g., simple opt-in email, simple opt-in phone, linkout, etc.) that may be delivered to users on the publisher's placement.In the alternative, another campaign type may be a retargeting cookiethat may be placed on a user's computer such that the user is delivereda particular advertisement when later browsing another website. In thealternative, a “data augmentation” campaign by an advertiser may seek toenhance an existing database of user information by requestingparticular information (e.g., the attributes that the inventioncollects, which may include attributes such as medical conditions,interests, whether the user would like to evaluate a new serviceprovider such as a cellular service provider, etc.) and recording thatinformation for the advertiser. Yet another campaign type may be amarket research campaign that seeks to determine responses to aparticular question or set of questions from a large number of users. Acampaign limits block 840 may permit a publisher to select the maximumnumber of times a visitor can see or convert on a campaign type in asingle visit to the publisher's placement. For example, a publishermight specify that a user may see a maximum of 8 simple opt-in emailcampaigns and convert a maximum of 6 such campaigns on a single visit.In this situation, if either threshold is reached, no further activityof the specified type will be permitted on the visit.

Referring now to FIGS. 9A and 9B, an exemplary layout for capturing oramending campaign settings may be provided. Settings block 910 may beprovided to capture a campaign name, a preferred rate type for thecampaign, a vertical to which the campaign is relevant (e.g., retail),one or more primary contact methods for contacting users, and anindication of whether it is desirable to collect a visitor's name.Visitor rules block 920 may be used to capture optional rules related toa campaign, for example, the maximum number of times that a campaign maybe viewed by a user per day, a maximum number of times that a user canconvert on the campaign in a lifetime or a particular time period, andwhether user verification is required before accepting a user as a lead.Budget block 930 may be used to capture budgeting thresholds for acampaign, including options to indicate whether a budget is present orwhether unlimited spending is permitted, whether the campaign hashourly, daily, monthly, or lifetime budgets, etc. It is desirable topermit daily, monthly, and lifetime budgets simultaneously in the eventthat an advertiser may wish to spread out advertising over days ormonths but still retain an overall budget. For example, an advertisermay set a daily budget of $100,000, a monthly budget of $500,000, and alifetime budget of $2,000,000. In this instance, five days at themaximum daily budget would exhaust the entire monthly budget and fourmonths at the maximum monthly budget would exhaust the entire lifetimebudget. A schedule block 940 may be provided to capture a schedule for acampaign. The schedule may include start and/or end dates, whether thecampaign is active on all days at all times or whether the campaignshould only be available during a particular set of days and times. Andan action block 950 may be provided to capture an indication of whethera campaign should be active or paused and whether a campaign should bedeleted. Node j indicates the connection between FIGS. 9A and 9B.

With reference to FIG. 10 , a suitable environment 1000 for implementingvarious aspects of the disclosed subject matter includes a handheld,portable, or stationary computer 1002. The computer 1002 includes aprocessing unit 1004, a system memory 1006, a codec 1005, and a systembus 1008. The system bus 1008 couples system components including, butnot limited to, the system memory 1006 to the processing unit 1004. Theprocessing unit 1004 can be any of various available processors. Dualmicroprocessors and other multiprocessor architectures also can beemployed as the processing unit 1004.

The system bus 1008 can be any of several types of bus structure(s)including the memory bus or memory controller, a peripheral bus orexternal bus, and/or a local bus using any variety of available busarchitectures including, but not limited to, Industrial StandardArchitecture (ISA), Micro-Channel Architecture (MSA), Extended ISA(EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB),Peripheral Component Interconnect (PCI), Card Bus, Universal Serial Bus(USB), and Advanced Graphics Port (AGP).

The system memory 1006 includes volatile memory 1010 and non-volatilememory 1012. The basic input/output system (BIOS), containing the basicroutines to transfer information between elements within the computer1002, such as during start-up, is stored in non-volatile memory 1012. Byway of illustration, and not limitation, non-volatile memory 1012 caninclude read only memory (ROM), programmable ROM (PROM), electricallyprogrammable ROM (EPROM), electrically erasable programmable ROM(EEPROM), or flash memory. Volatile memory 1010 includes random accessmemory (RAM), which acts as external cache memory. According to presentaspects, the volatile memory may store the write operation retry logic(not shown in FIG. 10 ) and the like. By way of illustration and notlimitation, RAM is available in many forms such as static RAM (SRAM),dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM(DDR SDRAM), enhanced SDRAM (ESDRAM).

Computer 1002 may also include removable/non-removable,volatile/non-volatile computer storage media. FIG. 10 illustrates, forexample, a disk storage 1014. Disk storage 1014 includes, but is notlimited to, devices like a magnetic disk drive, solid state disk (SSD),flash memory card, or memory stick. In addition, disk storage 1014 caninclude storage media separately or in combination with other storagemedia including, but not limited to, an optical disk drive. Tofacilitate connection of the disk storage devices 1014 to the system bus1008, a removable or non-removable interface is typically used, such asinterface 1016.

It is to be appreciated that FIG. 10 describes software that acts as anintermediary between users and the basic computer resources described inthe suitable operating environment 1000. Such software includes anoperating system 1018. Operating system 1018, which can be stored ondisk storage 1014, acts to control and allocate resources of thecomputer system 1002. Applications 1020 take advantage of the managementof resources by operating system 1018 through program modules 1024, andprogram data 1026, such as the boot/shutdown transaction table and thelike, stored either in system memory 1006 or on disk storage 1014. It isto be appreciated that the claimed subject matter can be implementedwith various operating systems or combinations of operating systems.

A user enters commands or information into the computer 1002 throughinput device(s) 1028. Input devices 1028 include, but are not limitedto, a touch screen, a pointing device such as a mouse, trackball,stylus, touch pad, keyboard, microphone, joystick, game pad, satellitedish, scanner, TV tuner card, digital camera, digital video camera, webcamera, and the like. These and other input devices connect to theprocessing unit 1004 through the system bus 1008 via interface port(s)1030. Interface port(s) 1030 include, for example, a serial port, aparallel port, a game port, and a universal serial bus (USB). Outputdevice(s) 1036 use some of the same type of ports as input device(s)1028. Thus, for example, a USB port may be used to provide input tocomputer 1002, and to output information from computer 1002 to an outputdevice 1036. Output adapter 1034 is provided to illustrate that thereare some output devices 1036 like monitors, speakers, and printers,among other output devices 1036, which might require special adapters.The output adapters 1034 include, by way of illustration and notlimitation, video and sound cards that provide a means of connectionbetween the output device 1036 and the system bus 1008. It should benoted that other devices and/or systems of devices provide both inputand output capabilities such as remote computer(s) 1038.

Computer 1002 can operate in a networked environment using logicalconnections to one or more remote computers, such as remote computer(s)1038. The remote computer(s) 1038 can be a personal computer, a server,a client, a processing center, a certificate authority, a router, anetwork PC, a workstation, a microprocessor based appliance, a peerdevice, a smart phone, a tablet, or other network node, and typicallyincludes many of the elements described relative to computer 1002. Forpurposes of brevity, only a memory storage device 1040 is illustratedwith remote computer(s) 1038. Remote computer(s) 1038 is logicallyconnected to computer 1002 through a network interface 1042 and thenconnected via communication connection(s) 1044. Network interface 1042encompasses wire and/or wireless communication networks such aslocal-area networks (LAN) and wide-area networks (WAN) and cellularnetworks. LAN technologies include Fiber Distributed Data Interface(FDDI), Copper Distributed Data Interface (CDDI), Ethernet, Token Ringand the like. WAN technologies include, but are not limited to,point-to-point links, circuit switching networks like IntegratedServices Digital Networks (ISDN) and variations thereon, packetswitching networks, and Digital Subscriber Lines (DSL).

Communication connection(s) 1044 refers to the hardware/softwareemployed to connect the network interface 1042 to the bus 1008. Whilecommunication connection 1044 is shown for illustrative clarity insidecomputer 1002, it can also be external to computer 1002. Thehardware/software necessary for connection to the network interface 1042includes, for exemplary purposes only, internal and externaltechnologies such as, modems including regular telephone grade modems,cable modems and DSL modems, ISDN adapters, and wired and wirelessEthernet cards, hubs, and routers.

Referring now to FIG. 11 , there is illustrated a schematic blockdiagram of a computing environment 1100 in accordance with the subjectspecification. The system 1100 includes one or more client(s) 1102,which can include an application or a system that accesses a service onthe server 1104. The client(s) 1102 can be hardware and/or software(e.g., threads, processes, computing devices). The client(s) 1102 canhouse cookie(s) and/or associated contextual information by employingthe specification, for example.

The system 1100 also includes one or more server(s) 1104. The server(s)1104 can also be hardware or hardware in combination with software(e.g., threads, processes, computing devices). The servers 1104 canhouse threads to perform, for example, identifying morphologicalfeatures, extracting meaning, auto generating FAQs, ranking, etc. Onepossible communication between a client 1102 and a server 1104 can be inthe form of a data packet adapted to be transmitted between two or morecomputer processes where the data packet contains, for example, acertificate. The data packet can include a cookie and/or associatedcontextual information, for example. The system 1100 includes acommunication framework 1106 (e.g., a global communication network suchas the Internet) that can be employed to facilitate communicationsbetween the client(s) 1102 and the server(s) 1104.

Communications can be facilitated via a wired (including optical fiber)and/or wireless technology. The client(s) 1102 are operatively connectedto one or more client data store(s) 1108 that can be employed to storeinformation local to the client(s) 1102 (e.g., cookie(s) and/orassociated contextual information). Similarly, the server(s) 1104 areoperatively connected to one or more server data store(s) 1110 that canbe employed to store information local to the servers 1104.

The illustrated aspects of the disclosure may also be practiced indistributed computing environments where certain tasks are performed byremote processing devices that are linked through a communicationsnetwork. In a distributed computing environment, program modules can belocated in both local and remote memory storage devices.

The processes described above can be embodied within hardware, such as asingle integrated circuit (IC) chip, multiple ICs, an applicationspecific integrated circuit (ASIC), or the like. Further, the order inwhich some or all of the process blocks appear in each process shouldnot be deemed limiting. Rather, it should be understood that some of theprocess blocks can be executed in a variety of orders that are not allof which may be explicitly illustrated herein.

What has been described above includes examples of the implementationsof the present invention. It is, of course, not possible to describeevery conceivable combination of components or methods for purposes ofdescribing the claimed subject matter, but many further combinations andpermutations of the subject embodiments are possible. Accordingly, theclaimed subject matter is intended to embrace all such alterations,modifications, and variations that fall within the spirit and scope ofthe appended claims. Moreover, the above description of illustratedimplementations of this disclosure, including what is described in theAbstract, is not intended to be exhaustive or to limit the disclosedimplementations to the precise forms disclosed. While specificimplementations and examples are described herein for illustrativepurposes, various modifications are possible that are considered withinthe scope of such implementations and examples, as those skilled in therelevant art can recognize.

In particular and in regard to the various functions performed by theabove described components, devices, circuits, systems and the like, theterms used to describe such components are intended to correspond,unless otherwise indicated, to any component which performs thespecified function of the described component (e.g., a functionalequivalent), even though not structurally equivalent to the disclosedstructure, which performs the function in the herein illustratedexemplary aspects of the claimed subject matter. In this regard, it willalso be recognized that the various embodiments includes a system aswell as a computer-readable storage medium having computer-executableinstructions for performing the acts and/or events of the variousmethods of the claimed subject matter.

What is claimed is:
 1. A data processing method, comprising: receiving,with a server communicative with a distributed streaming data pipelinefor communication with asynchronous event processing services, anindication of user identification data and an indication of a datasource; determining, based on the indication of the data source andretrieval of information related to the data source from a memoryassociated with a database server, an inquiry rule, wherein the inquiryrule defines an inquiry threshold; comparing, with the server, aninquiry count to the inquiry threshold; and sending, using the server,an inquiry for the user to the data source, wherein sending the inquiryfor the user comprises: receiving a plurality of indications of expectedutility of an inquiry, wherein one of the plurality of indications ofexpected utility is calculated by an event processing service based onthe utility of an inquiry response, the probability that the user willrespond to the inquiry, the user's expected response to the inquiry, andutility sharing percentage, comparing, using the server, the pluralityof indications of expected utility, and based on the comparing theplurality of indications of expected utility, transmitting with theserver an inquiry to the data source.
 2. The method of claim 1 furthercomprising: incrementing, with the server, the inquiry count; if theincremented inquiry count is less than or equal to the inquirythreshold, repeating the step of sending an inquiry; and if theincremented inquiry count is greater than the inquiry threshold,transmitting an indication of no further inquiries to the data source.3. The method of claim 1, wherein the inquiry comprises a question forthe user to answer or an advertisement for the user to view.
 4. Themethod of claim 1, further comprising: calculating a difference betweenthe inquiry count and the inquiry threshold; and if the difference isgreater than or equal to a drop-off threshold, determining that theinquiry comprises a question for the user to answer, and if thedifference is less than a drop-off threshold, determining that theinquiry may comprise a question for the user to answer or anadvertisement for the user to view.
 5. The method of claim 1, whereinthe user's expected response to the inquiry is based on past responsesto the inquiry by a plurality of prior users.
 6. The method of claim 1,wherein the plurality of indications of expected utility comprises anexpected drop-off rate based on the rate at which a plurality of priorusers discontinued responding to inquiries.
 7. The method of claim 6,wherein the plurality of indications of expected utility furthercomprises an indication of the number of inquiries to which a user mustrespond before any utility is recognized.
 8. The method of claim 1,wherein the inquiry comprises an advertisement for the user to view; andthe plurality of indications of expected utility comprises an expectedrate at which the user will take an action upon viewing theadvertisement, and wherein the expected rate is based on the rate atwhich a plurality of prior users took the action upon viewing theadvertisement.
 9. A data processing system, comprising: a computationnetwork configured to: receive, with a server communicative with adistributed streaming data pipeline for communication with asynchronousevent processing services, an indication of user identification data andan indication of a data source; determine, based on the indication ofthe data source and retrieval of information related to the data sourcefrom a memory associated with a database server, an inquiry rule,wherein the inquiry rule defines an inquiry threshold; compare, with theserver, an inquiry count to the inquiry threshold; and send, using theserver, an inquiry for the user to the data source, wherein sending theinquiry for the user comprises: receiving a plurality of indications ofexpected utility of an inquiry, wherein one of the plurality ofindications of expected utility is calculated by an event processingservice based on the utility of an inquiry response, the probabilitythat the user will respond to the inquiry, the user's expected responseto the inquiry, and utility sharing percentage, comparing, using theserver, the plurality of indications of expected utility, and based onthe comparing the plurality of indications of expected utility,transmitting with the server an inquiry to the data source.
 10. Thesystem of claim 9, wherein the computation network is further configuredto: increment, with the server, the inquiry count; if the incrementedinquiry count is less than or equal to the inquiry threshold, repeat thestep of sending an inquiry; and if the incremented inquiry count isgreater than the inquiry threshold, transmit an indication of no furtherinquiries to the data source.
 11. The system of claim 9, wherein theinquiry comprises a question for the user to answer or an advertisementfor the user to view.
 12. The system of claim 9, wherein the computationnetwork is further configured to: calculate a difference between theinquiry count and the inquiry threshold; and if the difference isgreater than or equal to a drop-off threshold, determine that theinquiry comprises a question for the user to answer, and if thedifference is less than a drop-off threshold, determine that the inquirymay comprise a question for the user to answer or an advertisement forthe user to view.
 13. The system of claim 10, wherein the user'sexpected response to the inquiry is based on past responses to theinquiry by a plurality of prior users.
 14. The system of claim 11,wherein the plurality of indications of expected utility comprises anexpected drop-off rate based on the rate at which a plurality of priorusers discontinued responding to inquiries.
 15. A non-transientcomputer-readable medium storing instructions that cause a systemcomprising a processor to perform operations, comprising: receiving,with a server communicative with a distributed streaming data pipelinefor communication with asynchronous event processing services, anindication of user identification data and an indication of a datasource; determining, based on the indication of the data source andretrieval of information related to the data source from a memoryassociated with a database server, an inquiry rule, wherein the inquiryrule defines an inquiry threshold; comparing, with the server, aninquiry count to the inquiry threshold; and sending, using the server,an inquiry for the user to the data source, wherein sending the inquiryfor the user comprises: receiving a plurality of indications of expectedutility of an inquiry, wherein one of the plurality of indications ofexpected utility is calculated by an event processing service based onthe utility of an inquiry response, the probability that the user willrespond to the inquiry, the user's expected response to the inquiry, andutility sharing percentage, comparing, using the server, the pluralityof indications of expected utility, and based on the comparing theplurality of indications of expected utility, transmitting with theserver an inquiry to the data source.
 16. The medium of claim 15,storing further instructions that cause a system comprising a processorto perform operations, comprising: incrementing, with the server, theinquiry count; if the incremented inquiry count is less than or equal tothe inquiry threshold, repeating the step of sending an inquiry; and ifthe incremented inquiry count is greater than the inquiry threshold,transmitting an indication of no further inquiries to the data source.17. The medium of claim 15, wherein the inquiry comprises a question forthe user to answer or an advertisement for the user to view.
 18. Themedium of claim 15, storing further instructions that cause a systemcomprising a processor to perform operations, comprising: calculating adifference between the inquiry count and the inquiry threshold; and ifthe difference is greater than or equal to a drop-off threshold,determining that the inquiry comprises a question for the user toanswer, and if the difference is less than a drop-off threshold,determining that the inquiry may comprise a question for the user toanswer or an advertisement for the user to view.
 19. The medium of claim16, wherein the user's expected response to the inquiry is based on pastresponses to the inquiry by a plurality of prior users.
 20. The mediumof claim 17, wherein the plurality of indications of expected utilitycomprises an expected drop-off rate based on the rate at which aplurality of prior users discontinued responding to inquiries.